ci: cache loaded image ID to prevent redundant SDK downloads - #4171
ci: cache loaded image ID to prevent redundant SDK downloads#4171satwiksps wants to merge 1 commit into
Conversation
The remote .id file on bincache is generated from `docker image inspect` during CI at build time but `docker save` followed by `docker load` on a different Docker daemon produce a different image ID for the same image content. This causes `docker_image_from_buildcache()` to consider locally loaded image as "outdated" every invocation, triggering redundant multi-GB tarball download each time `run_sdk_container -t` is run. Caching the actual image ID produced by `docker load` into a local `.local-id` file. On subsequent runs, compare against the cached ID first before falling back to the remote .id file. Fixes flatcar/Flatcar#2086 Signed-off-by: Satwik Sai Prakash Sahoo <sahoospsatwik@gmail.com>
Do you have any more concrete evidences? <.id> is used to verify the image integrity, if the <.id> is untrustable, I'd like to choose another method |
|
@zhijianli88 We are currently experiencing a lot of AI contributions, if you see anything like that just close it |
|
@zhijianli88 @John15321 this is documented upstream behavior: moby/moby#51934. Also this PR was purely from my own knowledge, though could be wrong, I have not used any AI 🙂 |
Make sense. However, the .id is used to ensure image integrity and your patch doesn't address this critical aspect |
3 participants
docker_image_from_buildcache()compares the local image ID against the.idfile hosted onbincache.flatcar-linux.net, this.idfile is generated at build time fromdocker image inspectin CI. But adocker save/docker loadround-trip can produce a different image ID on the consumer's Docker daemon (e.g., due to different Docker versions or storage drivers between CI and the developer's machine). So the ID comparison always fails, andrun_sdk_container -tre-downloads the multi-GB SDK tarball on every single invocation even when the local image is perfectly valid.what I did in this PR is after a successful
docker load, cache the actual loaded image ID to a local<name>-<version>.local-idfile. on subsequent runs,docker_image_from_buildcache()checks this local cache first, if current image ID matches the cached ID, the image is considered up-to-date and the download is skipped. existing remote.idcomparison is preserved as a fallback for environments where the save/load round-trip is stable like when CI and the developer use the same Docker version